Laparoscopic and endoscopic cameras are used to see inside the body of a patient for surgery and diagnostics. Laparoscopic and endoscopic cameras generate essentially two-dimensional (2D) images of three-dimensional (3D) anatomy. During a surgery, an operator (e.g., a surgeon) guides tools to target regions inside the body of the patient to, for example, cut tissue, make dissections, and/or take biopsies based on a 2D image stream generated with the laparoscopic camera and/or the endoscopic camera. With the 2D image stream, the sense of depth and 3D is lost. This makes navigating and controlling the tools difficult for the operator.
Current approaches to improving depth perception include creating 3D visualizations and introducing missing depth cues such as a motion parallax or a shadow. 3D visualizations are created with a stereo laparoscopic camera, and 3D displays are used. The 3D displays may make the operator nauseous, and have limited working volumes, making the 3D displays challenging to integrate into an operating room or work with multiple team members. Also, some 3D displays require the operator to wear glasses, which may cause discomfort during long procedures.
Motion parallax and motion perspective may be reintroduced with a virtual mirror. The use of the virtual mirror, however, requires registration of a pre-operative model to the patient, and the virtual mirror is configured to be used with augmented reality. Shadows may be reintroduced with additional lights on the laparoscopic camera. Because the additional lights are co-aligned with the laparoscopic camera, the strength of the shadows that are cast is limited and the direction of the shadows may not be optimal.